
We claim: 

1. A method for segmenting an image formed by a plurality of pixels using a 
region-merging process characterised by using covariance data and a plurality of vector 
components of each said pixel to evaluate a merging criterion for regions of said image. 

2. A method according to claim 1 wherein said plurality of vector components 
comprise at least two of colour, range and motion. 

3. A method according to claim 2 wherein said colour vector component comprises 
at least one colour channel of a colour space in which said image can be reproduced. 

4. A method for segmenting an image formed by a plurality of pixels, each said 
pixel being described by a vector having components each relating to a different 
measured image characteristic, said method comprising the steps of: 

(a) receiving, for each said pixel, a plurality of said vector components and 
a corresponding error covariance representation of said pixel; 

(b) for each said pixel, fitting each said component and the corresponding 
covariance representation to a predetermined linear model to obtain a set of model 
parameters and corresponding confidence representations; 

(c) statistically analysing the sets of model parameters and corresponding 
confidence representations to derive a segmentation of said image that minimises a 
predetermined cost function. 

5. A method according to claim 4 wherein step (c) comprises the sub-steps of: 



565702US-doc 



w . 24 - ^ 

(ca) defining said pixels to each be initial regions of said image; 

(cb) merging said regions in a statistical order using said sets of model 
parameters and confidence representations to obtain a null segmentation of said image; 

(cc) analysing a curve formed using said model parameters and 
corresponding confidence representations to determine an optimal halting criterion at 
which to cease the merging of said regions; and 

(cd) processing said merging of said initial regions to halt when said optimal 
merging criterion is reached. 

6. A method according to claim 5 wherein sub-step (cd) comprises re-executing the 
entire merge of said initial regions using said model parameters and confidence 
representations to provide said merged segmentation. 

7. A method according to claim 5 wherein sub-step (cc) comprises identifying 
returns to monotonicity from local minima in said curve and selecting a predetermined 
said return approaching the null segmentation as said optimal halting criterion. 

8. A method according to claim 7 wherein step (cd) comprises re-executing the 
merge of said regions using said model parameters up until said predetermined return is 
reached to provide said merged segmentation. 

9- A method according to claim 5 wherein said statistical order is determined using 

an order of minimum covariance-normalised vector distance between adjacent regions of 
said segmentation. 
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10. A method according to claim 5 wherein, said statistical order is determined using 
a length of a common boundary between adjacent regions. 



11. A method according to claim 5 wherein said statistical order is determined by 
dividing a minimum covariance-normalised vector distance between adjacent regions of 
said segmentation by a length of a common boundary between adjacent regions, and 
ordering the resulting quotients. 

12. A method according to claim 1 1 wherein each said quotient forms a test statistic, 
a record of which is retained at each merging step. 

13. A method according to claim 4, wherein said plurality of vector components 
comprise at least two of colour, range and motion. 

14. A method according to claim 5, wherein said colour vector component comprises 
at least one colour channel of a colour space in which said image can be reproduced. 

15. A method for unsupervised selection of a stopping point for a region-merging 
segmentation process, said method comprising the steps of: 

(a) analysing a graph of merging cost values to identify departures from 
substantial monotonicity of said graph; and 

(b) selecting said stopping point to be a merging cost value corresponding 
to a return to monotonicity of said graph, said selected stopping point being associated 
with one of a limited plurality of final said departures in said region merging process. 
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16. A method according to claim 15 wherein said selected stopping point comprises 
a return from s^id final departure. 



17. A method according to claim 15 wherein said departures are larger than a 
predetermined threshold. 

18. A method according to claim 15 wherein said merging cost function comprises 
an ordered series of test statistics, each said test statistic being formed, for each adjacent 
pair of regions in the segmented image, by dividing a covariance-noimalised vector 
distance between the pair by a length of a common boundary between the pair. 

19. Apparatus for segmenting an image formed by a plurality of pixels using a 
region-merging process characterised by using covariance data and a plurality of vector 
components of each said pixel to evaluate a merging criterion for regions of said image. 

20. Apparatus according to claim 19 wherein said plurality of vector components 
comprise at least two of colour, range and motion. 

2 1 . Apparatus according to claim 20 wherein said colour vector component 
comprises at least one colour channel of a colour space in which said image can be 
reproduced. 



22. Apparatus for segmenting an image formed by a plurality of pixels, each said 
pixel being described by a vector having components each relating to a different 
measured image characteristic, said apparatus comprising: 
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means for receiving, for each said pixel, a plurality of said vector components 
and a corresponding error covariance representation of said pixel; 

means for fitting, for each said pixel, each said component and the corresponding 
covariance representation to a predetermined linear model to obtain a set of model 
parameters and corresponding confidence representations; and 

analysing means for statistically analysing the sets of model parameters and 
corresponding confidence representations to derive a segmentation of said image that 
minimises a predetermined cost function. 

23. Apparatus according to claim 22 wherein said analysing means comprises: 

defining means for defining said pixels to each be initial regions of said image; 
merging means for merging said regions in a statistical oixier using said sets of 

model parameters and confidence representations to obtain a null segmentation of said 

image; 

curve analysing means for analysing a curve formed using said model parameters 
and corresponding confidence representations to determine an optimal halting criterion at 
which to cease the merging of said regions; and 

processing means for processing said merging of said initial regions to halt when 
said optimal merging criterion is reached. 

24. Apparatus according to claim 23 wherein said processing means comprises 
means for re-executing the entire merge of said initial regions using said model 
parameters and confidence representations to provide said merged segmentation. 
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25. Apparatus according to claim 23 wherein said cuive analysing means comprises 
means for identifying returns to monotonicity from local minima in said curve and means 
for selecting a predetermined said return approaching the null segmentation as said 
optimal halting criterion. 

26. Apparatus according to claim 25 wherein said processing means comprises 
means for re-executing the merge of said regions using said model parameters up until 
said predetermined return is reached to provide said merged segmentation. 

27. Apparatus according to claim 23 wherein said statistical order is determined 
using an order of minimum covariance-normalised vector distance between adjacent 
regions of said segmentation. 

28. Apparatus according to claim 23 wherein said statistical order is determined 
using a length of a common boundary between adjacent regions, 

29. Apparatus according to claim 23 wherein said statistical order is determined by 
dividing a minimum covariance-normalised vector distance between adjacent regions of 
said segmentation by a length of a common boundary between adjacent regions, and 
ordering the resulting quotients. 

30. Apparatus according to claim 29 wherein each said quotient forms a test statistic, 
a record of which is retained at each merging. 
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3 1 . Apparatus according to claim 22, wherein said plurality of vector components 
comprise at least two of colour, range and motion. 

32. Apparatus according to claim 23, wherein said colour vector component 
comprises at least one colour channel of a colour space in which said image can be 
reproduced. 

33* Apparatus for unsupervised selection of a stopping point for a region-merging 
segmentation process, said apparatus comprising: 

means for analysing a graph of merging cost values to identify departures from 
substantial monotonicity of said graph; and 

means for selecting said stopping point to be a merging cost value corresponding 
to a return to monotonicity of said graph, said selected stopping point being associated 
^with one of a limited plurality of final said departures in said region merging process. 

34. Apparatus according to claim 33 wherein said selected stopping point comprises 
a return from said final departure- 

35. Apparatus according to claim 33 wherein said departures are larger than a 
predetermined threshold. 

36. Apparatus according to claim 33 wherein said merging cost function comprises 
an ordered series of test statistics, each said test statistic being formed, for each adjacent 
pair of regions in the segmented image, by dividing a covariance-normalised vector 
distance between the pair by a length of a common boundary between the pair. 
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37. A program for making a computer execute a procedure to segment an image 
formed by a plurality of pixels using a region-merging process characterised by using 
covariance data and a plurality of vector components of each said pixel to evaluate a 
merging criterion for regions of said image. 

38. A program according to claim 37 wherein said plurality of vector components 
comprise at least two of colour, range and motion. 

39. A program according to claim 38 wherein said colour vector component 
comprises at least one colour channel of a colour space in which said image can be 
reproduced. 

40. A program for making a computer execute a procedure to segment an image 
formed by a plurality of pixels, each said pixel being described by a vector having 
components each relating to a different measured image characteristic, said program 
comprising: 

code for receiving, for each said pixel, a plurality of said vector components and 
a corresponding error covariance representation of said pixel; 

code for, for each said pixel, fitting each said component and the corresponding 
covariance representation to a predetermined linear model to obtain a set of model 
parameters and corresponding confidence representations; and 

analysing code for statistically analysing the sets of model parameters and 
corresponding confidence representations to derive a segmentation of said image that 
minimises a predetermined cost function. 
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41 A program according to claim 40 wherein said analysing code comprises: 
code for defining said pixels to each be initial regions of said image; 
code for merging said regions in a statistical order using said sets of model 

parameters and confidence representations to obtain a null segmentation of said image; 

code for analysing a curve formed using said model parameters and 

corresponding confidence representations to determine an optimal halting criterion at 

which to cease the merging of said regions; and 

code for processing said merging of said initial regions to halt when said optimal 
merging criterion is reached. 

42. A program for making a computer execute a procedure for unsupervised 
selection of a stopping point for a region-merging segmentation process, said program 
comprising: 

code for analysing a graph of merging cost values to identify departures ftom 
substantial monotonicity of said graph; and 

code for selecting said stopping point to be a merging cost value corresponding 
to a return to monotonicity of said graph, said selected stopping point being associated 
with one of a limited plurality of final said departures in said region merging process. 

43. A program according to claim 42 wherein said selected stopping point comprises 
a return from said final departure. 

44. A program according to claim 43 wherein said departures are larger than a 
predetermined threshold. 



45. A program according to claim 42 wherein said merging cost function comprises 
an ordered series of test statistics, each said test statistic being formed, for each adjacent 
pair of regions in the segmented image, by dividing a covariance-normalised vector 
distance between the pair by a length of a common boundary between the pair. 
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